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ABSTRACT 

This study presents a quantitative method for scoring 
concept maps generated by students learning int.roductory college 
chemistry. Concept maps measure the amount of chenical information 
the student possesses, reasoning ability m chemistry, and specific 
misconceptions about introductory and physical chemistry concepts. 
They provide a visualization of cognitive structure. When a student 
draws a concept map for chemical reactions, the result is a model of 
the student's conceptual framework for understanding the concepts and 
propositions of chemical change. Developing a valid method for 
scoring student concept maps will enable educators to evaluate 
student knowledge free of the bias and arbitrariness often associated 
with qualitative reviews. Concept saps nay be evaluated 
quantitatively by categories. The category score for prepositional 
validity reflects student reasoning ability in chemistry. The score 
Significantly correlates with formal reasoning ability in chemistry. 
The category score for hierarchical structure reflects the amount of 
chemical information possessed by a student, students who pu.sess 
large amounts of information about chemistry, position more 
vocabulary words within each hierarchical level than the student who 
demonstrates limited chemical knowledge. It is suggested that .he 
greater a student's understanding of introductory chemistry concepts 
I . f!!®/*™'*^ employed in mapping concepts and propositions 
related to chemical reactions. Low strand count reflected specific 
misconceptions about Avogadro's Number, the mole concept, and the Law 
of conservation of Matter. (Author/HM) i^w 
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Introductory Caiimg^ ChgfnUtrY 
ABSTRACT 

This study presents a quantitatlvt method for scoring concept maps generated by 
students leamjng Introductory college chemistry. Concept maps provide a visualization 
of cognitive structure. When a student draws a concept map for chemteal reactions the 
result Is a personification of the students conceptual framework for understanding the 
concepts and propositions of chemical change. Developing a valid method for scoring 
student concept maps will enable educators to evaluate student knowledge free of the bias 
and arbitrariness often associated with qualitative reviews. 

The scoring methodology presented In this study permits evaluatfon of the learning 
characteristics of students In chemistry. The concept map measures the amount of 
chemical Informatton the student possesses, reasoning ability In chemistry, and specific 
misconceptions about Introductory and physfcal chemistry concepts. 

This research has demonstrated that concept maps may be evaluated quantitatively 
by categories. The category score for prepositional validity reflects student reasoning 
ability In chemistry. Prepositional validity Is measured by the ratto of valkj connecting 
lines to total number of connecting lines drawn {per strand). This score significantly 
correlated with formal reasoning ability In chemistry. 

A second category score was klentified for hierarchical structure. The category 
score for hierarchical structure reflects the amount of chemical Information possessed 
by a student. Students who possess large amounts of Information about chemistry 
position more vocabulary words within each hierarchical level th; . the student who 
demonstrates limited chemical knowledge. 
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The data in this research also suggest the greater a student's understanding of 
introductory chemistry concepts the more stramte s/he employs In mapping concepts and 
propositions related to chemical reactions. Low strand count reflected ^cific 
misconceptions about Avogadro's Numl>er. the mole concept, and the Law of Consen^atbn 
of Matter. 

This study documents a method for objectively measuring student heuristic 
processing in chemistry. Educators previously have used the concept map as an 
instructional tool and as a metaleaming strategy. Successful quantitative evaluation of 
the concept map enables it to be used as a diagnostte tool to monitor and explore 
conceptual change. 

One application of this study's findings is the utilization of concept map category 
scores to make informed decisions about Instructional design. This will enhance rather 
than hinder student reconceptualizatlon. Educators in science may consult student 
concept map category scores to gain Infonnation alvMjt student understanding in a given 
domain. Effective educational programs which strive to remediate misconceptions about 
scientific knowledge may use this Information to develop Instructional strategies 
complementary to Individual student learning traits. Utilizing category scores as a 
student diagnostic tool prior to cun-iculum development Is appropriate for live, teacher- 
assisted classroom Instruction as well as knowledge-based computer-assisted 
Instruction. 
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SfisulDflSliujam-Gengratftti ganfigalMfliLiilii 

introductory Colloqe ChomiBtry 



INTRODUCTION 

A concept map is an hierarchicai arrangement of coi%ept names and conceptual 
relationships (Novait and Gowin, 1984). Researchers have used the student-generated 
concept map as a diagnostk; tod to evaluate student understanding of science (Moreira, 
1987; Ault, Novak and Gowin, 1988; Brody, 1989). Prevbus analyses of students' 
maps have i^een primarily qualitative. Researchers have attempted to detemiine what it 
is the student icnows, or thinks s/he knows, by discussing the student's map during 
clinical interviews. Ankney and Joyce (1974) claimed that interviews with students are 
a source of variance however. 

Many researchers believe the student-prepared, (tomain-sii^fic concept map is a 
viable tool for interpretation of student understanding in physics, biology, and 
chemistry (Moreira. 1987; Cleare, 1983; Brody, 1989). A concensus Is missing 
however, on how to evaluate the novice m£^. Some educators have devetoped scoring 
protocols for student-generated concept maps (Cleare, 1983; Wallace and Mintzes, 
1990). The methodotogy requires a map evaluator to award points for particular map 
characteristics. Research by Smith (1975), Ausubei (1968). Alverez and Risko 
(1987), Novak and Gowin (1984), and Brody (1989) suggest that the cognitive 
processes of categorizatton, class includton, progressive differentiation, and integrath/e 
reconciliation are represented by a student's concept map. Successful quantitative 
evaluatton of student concept maps may reveal significant information about student 
cognitive orientation toward learning science. 



This study deswibss a quantitative scoring methoctotogy supported by current 
teaming theory. The topic of chemica! reactions (with concepts and pioposHtons related 
to chemical change) was selected because It provides a primary focus In first year 
college chemistry courses. Burton (1988) suggested a domain-specific concept ir.^ is a 
product of the preparer's decision-making processes employed t team that subject. 
Consequently, this study hypothesized that scores of ooncei;^ maps for chemical reactions 
reflect traits students exhtoit when learning chemistry. 

METHOD 

This study Investigated the association between shjdent concept map scores In 
chemistry and learning traits (Independent variables). The amount of content 
Information a student possesses and the level of reasoning ability a student exhibits in 
the domain are examples of characteristics which Influence meaningful learning of 
chemistry. It is possible that one formal reasoner In chemistry may design a map 
incorporating multiple strands and few, yet sophistfeated propositions. Another formal 
reasoner however, may embed many concepts within one major hierarchy, not 
categorizing into simpler, separately functioning "mlnl-hierarchles" (strands). In 
contrast, a third student, a concrete reasoner, may prepare a concept map much less 
intricate than either of these examples. 

Data was collected over a period of e^ht weeks in the fall semester of 1989. Sixty- 
five students partk^ipated in the study. £^ student was enrolled m a tour year technteal 
engineering college program. AM subjects initially were Inexperienced with concept 
mapping technique. All students participated In lnlroduc:ory activities, mapping 
concepts related to their engineering degrees. Sixty-one of the 65 participants were 
male and ranged in age from 19 to 26 years. 
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upon completion of the introductory exercises, students were instructed to map a 
{determined list of concepts related to diemlcai reaction equations and ^lemical diange 
(see Figure 1: Concept Maf^r^ in Chemistry ). The stutfent-generated concept mspB ki 
chemistry were then scored. During this same period, the subjects were assessed for 
chemistry content knovWedge, cognitive reasoning in science, preferred learning style, 
creative and critical ih5r:king, attitude toward chemistry, and demographic background. 

General misconctjpiions of content were measured by the American Chemistry 
Society's (ACS) chemi-jlry achievement test High School Chemistry -Form 1987. The 
Group Assessment of Logical Thinl^ing (GALT) evaluated student cognitive reasoning 
capability in science. A student's inherent teaming style was determined by Kolb's 
Learning Style inventory (LSI). Creative and critical thinklr^ skills were measured as a 
subtest of GALT. Wfunwright's Attitude Toward Chemistry (1985) established a student's 
affect for the s-jbject. Also, the subjects completed a questionnaire called 
individual/Demographic Data. 

An expert (criterion) map served as a template for scoring student-generated 
concept maps (see Figure 2: Bxpert Map ). A panel of three college chemistry professors 
developed the expert map. Evaluators of student maps compared novice hierarachlcal 
structures and propositions to those rep-esented by the experts* template. {See Figures 
3a and 3b for samples of student-generated maps.) Construct validity of student maps 
was determined by the degree of appropriateness of the hierarchfcal structures and 
propositions presented. 

The prescrited scoring methodology for evaluation of student-generated concept 
maps in chemistry focused on three components: (1) physical {hierarchk:al) structure 
of the map. (2) Identified proposittons. and (3) the actual validity versus Implied 
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validity of these components. Hierarchical structure and propositional validity were 
further defined and evaluated by examining specific map instructs. 

Student-generated concept maps were scored by first identifying the following niap 
oonstnK^ts (see Figure 4: Concept Msp Scoiir^ Form ): 

• Total Number of Vocabulary incorporated Into the Map 

• Number of Hierarchical L^eis 

• Ratio of Number of Vocabulary to Number of Hierarchicai Levels 

• Number of Connecting Unes Drawn 

• Number of Connect Unes Vaik^ labeled 

• Ratto of Connecting L fnes Valic% Li^ed to Number of Connecting 
Unes Drawn 

• Number of Strands Recognized Within the Map 

• Number o Cross- IJni(s Between Strands 

The overall concept map score X Is represented mathematicany with the formula 
adjusted for discrete cognitive structuring differences. X a [x-ii(b«e)] ♦ b/e, where: 

X = initial tally of points (ratios) awarded for recognition of 
hierarachical. propositional and validity constructs on 
concept map. 

n s numt>er of strands in conc^t ms^. 

b B summed ratios of number of vocabulary terms to numl>er of 

hierarchical levels (per strand}, 
e B summed ratios of number of valkj connecting lines to total 

number of comectlng lines drawn. 

The term '*n(b4e}*' adjusted the initial concept map point tally to accommodate 
scoring maps in which students incorporated multiple strands that were weak 
conceptually or developed poorly. The ratio "b/c" tabulates strong stuttent proficiencies 
in categorization and progressh^e differentiation si^ilis. Students who incorporate an 
extensive number of vocabulary terms per hierarchical level, and illustrate a strong 
ratio of valid connecting lines to total number of lines drawn, acquire additional points. 
Map scores for chemical reactions were then correlated to the independent variables 
u»ng ANOVA. 
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RESULTS 

Overall concept map scores and oonce;^ map category scores were compared to the 
validated measures of student learning traits in cliemlstry. Significant correlations 
resuited. 

Overall Concept Map Scores 

Anmnl oi Chfimicai infprmaiion Possessed fixSmdsni and Smdanl gflODllbtf 
Reasoning Ability : Significant positive correlation existed between standardized ACS 
chemistry test scores and students' overall con{^pt map scores. Results suggested that 
the higher the overall concept map score for chemical reactions the greater the amount 
of chemical Itnowiedge possessed. Analysis of variance In overall concept map score fo 
GALT categorical data (formal, transitional and corKsrete reasoning) also indicated the 
greater the ovdrail concept map score the more formal the student's cognitive ability In 
science. See Tablet and Table 2. (The study ranked standardized scores to allow for 
analysis of variance of between-group and wHhin-group means.) 



Table 1 

One Factor ANOVA Xi: ACS Category Yv- CM Formuiated/Z 

Anatyste of Variance Table 



Source: 


DF: 


Sum Square: 


Mean Square: 


F-test: 


Between groupii 


4 


899.983 


224.996 


3.681 


Within groups 


59 


3605.955 


61.118 


p m .0096 


Total 


63 


4505.938 







Mode) II estimate of t)etween component variarwe » 40.969 



P.igeS 



10 



Table 2 

On» Factor ANOVA 

X ^ : Qatt Catesory * : CM Formuiatad/Z 
Y 

Comparison: Mean Piff.: Fisher PLSD: Schoffe F-test : 



Formal vs. Transitional 


10.772 


3.61 S* 


31.338* 


Formal vs. Concrete 


18.344 


5.129* 


45.175* 


Transitional vs. Concrete 


7.573 


5.219- 


7.434* 



* Significant at 99% 



Based on the Initial findings above, this study pot..ulated that high chemical 
knowledge scores reflect formal reasoning in chemistry. Further investigation was 
pursued to determine If an overall concept map score was simply a summation of the two 
traits. Analysis of variance In GALT score as correlated to chemical knowledge was 
significant (p » .0035). However, the variance In GALT was s^nificant only between 
the very high ACS scorers and all other partk;ipants. 

it appears that the highest overall concept map scores belonged to those students 
who reason formally about a very large amount of chemical Information. The lowest 
overall concept map scores belonged to those students who reason concretely about a very 
small amount of chemical Information. The students who reason formally about small 
amounts of chemical Information, and the students who rea^n concretely about large 
amounts of chemical iriformation were not significantly Identified by overall concept 
map scores. 

Student IsaEPina Slyis ia Chemistry : Overall scores of concept maps for chemical 
reactions were successful In Identifying two of the preferred learning styles 
investigated. Accommodators and assimilators in chemistry displayed significant 
differences In the construction of concept meps for chemical reactions. Students who 



Page 6 

11 



obtained a very high overall concept map s«>re prefer to learn chemistry through 
assimilation. The students who obtained a very low overall oonc^t map score prefer to 
learn chemistry througn accommodatton. A student's preferred learning style in a 
domain ctepends on the extent s/he integrates such behaviors as concrete experience, 
refiecth^e observation, abstract conceptualization, and active experimentation when 
learning. A student may be categorized an accommodator, diverger, converger, or 
assimitator (Kolb). 

Concept Map Category Scores 

Students wiio reason formally about a small amount of chemical informatton. and 
students who reason concretely about a la^e amount of chemical information were not 
identified by overall scores of concept maps for chemical reactions. Category scores of 
individual components of the concept map however did descrft>e these students' learning 
traits in chemistry. 

Proposltional Validity : Sixty-seven percent and 44% of the students who scored 
very high and high (respectively) in the map category for prepositional validity 
exhibited formal reasoning sidlis in chemistry. Eighty-one percent of the students who 
scored low and very low In the propositional validity category exhibited concrete 
reasoning in chemistry. Transitional reasoners in chemistry were not significantly 
identified by the concept map subscore of "prepositional validity." 

The ratio of number of validly labeled connecting lines to total number of 
connecting lines drawn measured the "propositional validity" of a concept map. Students 
who scored high in the category of propositional validity were significantly identified as 
formal reasoners in chemistry, independent of the amount of chemical information 
possessed. The concrete reasoners in diemistry illustrated low propositional validity on 
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concept maps for chemical reacttons. again independent of the amout of chemical 
information possessed. 

A high category score for prepositional validity also suggests that the map 
preparer prefers to learn chemistry through assimilation. The student appears to 
combine strengths of abstract conceptualization and reflective obsen^ation to organize a 
wide range of information into concise, togical fomi (Kob). This studenrs map displays 
large numbers of concepts emt>eckjed within few hierarchicai levels. Insightful arid 
creative associations between concepts are identifiable. Propositionai valkjity of this 
map is high. 

In contrast, the student who prefers to learn chemistry through BCComnK}dation 
scores low in the concept map category of propositionai valkJIty. This student seeks 
concrete experiences and active experimentation (Kolb). Data reveal that this map 
preparer uses only a few concept names in each hierarchical level and frequently omits 
verb/verb phrases on connecting lines. Relationships between concepts are embigix>us. 
Propositionai validity of this concept map is low. 

Hierarchical Ordering Within Sfil& fil CfiOiSfifiiS- The ratio of number of voci^lary 
utilized per hierarchicai level (per strand) represented the physical structure of the 
student's concept map for chemical reactions. Significant variance existed between 
students wtio possess a high amount of chemical knowledge and students who possess a low 
amount of chemical knowledge. Significant variance in the stru^ural component map 
score also existed between studente who possess middle and k?w levels of chemical 
Information. 

Analysis of map subsoores for structural ordering of concepts indicated that high 
scorers In this category were students who possess a greater amount oi chemical 
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Informatton. Forty-six percent of t^te students who scored high tor the ratio of number 
of vocabulary utiiizeJ per hierarchical level (per strand) scored high on the ACS 
chemistry test. A disproporttonate nunnber of stiMients who scored middle, low. and very 
low for the structural ordering component scored in the middle range for amount of 
chemical information possessed (47%, 46%, and 66.6%, respecth^ety). 

Further anatysis of student scores for the map category of structural ordering 
illustrated significant differences between stiKient subscores and specific types of 
chemical misconceptions. The students who scored high on the component nnap score 
'structural ordering* not only possess greater amounts of diemical informatbn, out the 
information possessed speciftcally represents introductory chemical concepts. 

Strand Count : The concept map subscore "strand '^unt" also identified f tudent 
understanding of Introductory Chemistry concepts. Data suggest the more strands a 
student employs to embed ordered sets of concepts and propcsHions within his/her 
concept map the greater the student's understanding of Introductory Chemistry concepts 
as measured by the ACS standardized test. (Test items which illustrated introductory 
Chemistry concepts Include questions about the law of Conservation of Mass; Avogadro's 
Number and the mole concept; and graphical representatbn of ph^'sicai change [cooling 
curve].) 

DISCUSSION 

The purpose of this study was to describe student understanding of chemical change 
by scoring student-generated concept maps tor chemical reactions. The scoring protocol 
devebped siK^cessfuliy described student learning dispositbn in chennistry. Quantitath^ 
evaluation of student-generated concept m^ enabes concept mapping to be used as a 
dlagnostb tool to nrivnltor and expbre conceptual change. 



Page 9 

J4 



Understanding how a student conceptualizes chemical ideas is necessary for 
successful remediation of misconceptions in chemistry. The ability to diagnose student 
understanding of chemical change is prerequisite to deveiopment of cumcuia for 
teaching chemical reactions and region e(^attons. A practk:ai af^ication of this study's 
findings is the utilization of concept map category scores to make informed decisions 
about instructionat design. 

Educators in science may consult student concept map category scores to gain 
information about stiident cognitive reasoning ability, amount of information possessed, 
and the student's pretbrred learning style in the ctomain. Efffective educational programs 
which strive to remediate misconceptions about scientific knowledge must use this 
information to develop instructional strategies complementary' to individual student 
leaning traits. Utilizing concept map ^tego^ scores as a studsnt diagnostic tool prior 
to curriculum development is appropriate for live, teacher-assisted classroom 
instruction as well as Knowledge-based computer-assisted Instructbn. 
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Figure 1 



Concept Mapping in Chemistry 

A. Below is a list of vocabulary related to chemical reactions, 
chemical reaction equations, and chemical change. 

B. Map these concepts to produce a visual configuration of how you 
think about the words and the relationships between them. 

C. A concept map represents conceptual organization of your 
thoughts and understanding of a subject. Remember to position 
object words and event words hierarchically, with more general, 
less specific concepts superordinate to more specific, less general 
ideas. Do not forget to clearly label all connecting lines: both cross- 
links between strands and prepositional links within strands. 



1 . 2HgO-^ 2Hg ♦ 

2. chemica! reactions 

3. neutralization 

4. 2Na 4- S -♦NajS 

5. combination/synthesis 

6. chemk^al reaction equatbns 

7. reactants 

8. valence number 

9. 2Ha Mg(OH)j MgCI^ -»■ 2HjO 

10. coefficients 

11. solutions 

12. Law of Conservation of Matter 

13. subscripts 

14. bases 

15. compounds 

16. double replacen>ent 

17. NaCi + ApriO*AgCI + NaNO, 



18. diemical formulas 

19. products 

20. rearrangement of matter 

21 . acids 

22. transferring electrons 

23. decomoosition 

24. CaO ♦ CO ♦ CaCO ^ 

25. fining ions^ 

26. relaiVe mole ratio 

27. ox ittotion- reduction 

28. single replacement 

29. separating molecules 

30. baiancint^ianced 

31. Fe ♦ H,SO-^FeS0 + H, 

24 4 * 

32. atoms 

33. elements 

34. oxidation state 

35. BaC» • 2H 0-^BaCI + 2H O 

22 22 



ID. # 
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Saxqpla X: Studbot'€ez»x«tad Ooooept HKp (Radial OonstnMtioa} 




J9 

BEST COPY MILACLI 



Saa^d XZ: StiKtent-G^ieratad CcHscept 





double ^ 


^single 










S ) (^CbO ♦ CC^^^^CaCC^^ (Nag ♦ AgNC;»'/^ » IW«> j) 




c 



oxidation- 
ruction 



mtunpktk 



(deconyosition) 



^scparatfog molectrias^ 



solution 



joxi 



involves cte^l^ 
in 



oxktotion 
states 
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Figure 4 



1. waiBEH OF mrwate: 



X DUMBER OF CBOSS.HMKS BETWEEN D1FFERIWQ STHANPS 



(Sfrmf rsoe^^d of^ It two w mom voc^u^ 



a. N9, gF YggA8VkARY 

{P0f Stran(5) 



(a) &rmd Oro : 

(b) 8ir»Ki 'nwo: , 

(c) Strand ThfM: 



(a) Ofwwn: 

(b) ValMly UbaMt 



a. MO. OP HiEBAftCMicAL^ LEVgLS BAT» OF HO, OF YPCABm QF H>6R, kEYELS 

(Per Stnuv}) ^ )«wM chi cal O^tiin^iom ^ to (Per Sirand) 



(a) S^and Ona: . 
^) Strand T^: ^ 
(c) Strami Threa: 



(a) S'mnd ^ia: 

(b) SHWKi Two: . 

(c) Straff Tlwaa: 



fi. MQ. QF CQWIgCTlMfi LIMES MAWW 

(Par l^rantQ 



(a) Stram} Ona: 

(b) Sirand Tvvo: 

(c) ^land Threa: 



ft>SVM OF RATIO'S 



L MQ COMMECTttiQ LiMgS VAL»LY^ LABELED 

(F^ S&'antQ *Va»tf»y aM^^^ ^ Compartsm ef 

Smctom-GeneraiBd Map W»h Cftoton lAa^. 



(a) Stmd Ona: 
^) StrarKi Two: 



(c) Strand Three: 



(a) CfOSS^JnWVaM : Drawi 

(b) Voc^ntory : Hierarchy L^ls (Si^im^hm o! Aii S^®^) 

(c) ConracUng Urm/Va9d : Ormm (Sumn^lim of AB Strar^fe} 



(d) Total Sum of Ratals fa' ♦ V ♦ tr* - x) 



11. TOTAL NUMBER OF ^EXAMPLES" tPENTIFgD: 



% » 



BATK? OF CONHECTtNQ ITNEy 



(a) Strand Ona: 

(b) Strand Jw: 



(c) Strand Three: 



Ifi. FrWAL COWCEPT MAP SCORE - 1 ^ 
X - Ix-n(b*c)J ♦ bfc • 
X* lOX^SO 

• v. *c' ara Y Defined Above (#9) 
'n* - Number of S^ancte (#1) 
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